{% extends "base.html" %}
{% block content %}
<section class="text-box">
  <img src="/static/indieweb.png" alt="IndieWeb logo" class="logo_image" />
  <h1 style="text-align: center;">About IndieWeb Search</h1>
  <p>Finding a blog post written by someone on the web is hard, especially if the author is new to publishing on the web and does not have much of an audience.</p>
  <p>Large search engines like Google and Bing have democratized access to information but they were not designed to help you find smaller, niche sites who may not think about SEO.</p>
  <p>This search engine lets you surf the IndieWeb, a community of people who own and make their own websites. There are other IndieWeb-adjacent sites indexed too.</p>
  <p>We do not expect that the first result will immediately answer a query you have. Rather, we see IndieWeb Search as an exploration tool so you can find websites and posts from a variety of sites that might interest you.</p>
  <p>IndieWeb Search provides special functionality for our wiki and the Microformats wiki. This special functionality lets us serve direct answers to queries like ("what is a h card" or "what is a reply?").</p>
  <p>The IndieWeb search engine is a work in progress. The source code is <a href="https://github.com/capjamesg/indieweb-search">available on GitHub</a> for those who want to contribute. We are particularly interested in improving our crawl efficiency and search results but if you see any opportunities for improvement we want to know about it.</p>
  <p>The crawler behind this search engine only crawls a portion of each website in the index. This ensures that we can index content from a vast range of sources.</p>
  <p>We hope you enjoy the IndieWeb search engine!</p>
  <h2>What user agent do you use?</h2>
  <p>Pages in this search engine are indexed using a crawler with the user agent "indieweb-search". The crawler obeys robots.txt directives so you can block our crawler from looking at certain pages on your site (or your whole site) if you would like.</p>
  <h2>Do you obey robots.txt directives?</h2>
  <p>Yes. IndieWeb Search obeys robots.txt files. This file is fetched before a sitemap or any page on a site is retrieved so long as the robots.txt file is available.</p>
  <h2>Can you index my site?</h2>
  <p>We are not currently accepting individual requests to index sites outwith the IndieWeb community.</p>
  <p>If you are a member of the community, add your domain name as an issue in the <a href="https://github.com/capjamesg/indieweb-search">project GitHub repository</a> and we may add you to our crawl list.</p>
  <h2>Will you recrawl my site?</h2>
  <p>We plan to recrawl sites on a regular basis. The logic for recrawling is being determined.</p>
  <h2>How can I ask you to index my newest content automatically?</h2>
  <p>Providing manual indexing requests on-demand is on our radar but has not been implemented. As a result, you will need to wait for us to crawl your site again before a new page is added.</p>
  <p>If you would like to see an automatic indexing API endpoint, <a href="https://github.com/capjamesg/indieweb-search"> let us know on GitHub</a>.</p>
  <p><a href="/">Click here to go to back to the IndieWeb search engine.</a></p>
</section>
{% endblock content %}